Skip to main content
SpringerLink
Log in

Envisioned speech recognition using EEG sensors

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Recent advances in EEG technology makes brain-computer-interface (BCI) an exciting field of research. BCI is primarily used to adopt with the paralyzed human body parts. However, BCI in envisioned speech recognition using electroencephalogram (EEG) signals has not been studied in details. Therefore, developing robust speech recognition system using EEG signals was proposed. In this paper, we propose a coarse-to-fine-level envisioned speech recognition framework with the help of EEG signals that can be thought of as a serious contribution in this field of research. Coarse-level classification is used to differentiate/categorize text and non-text classes using random forest (RF) classifier. Next, a finer-level imagined speech recognition of each class has been carried out. EEG data of 30 text and not-text classes including characters, digits, and object images have been imagined by 23 participants in this study. A recognition accuracy of 85.20 and 67.03% has been recorded at coarse- and fine-level classifications, respectively. The proposed framework outperforms the existing research work in terms of accuracy. We also show the robustness in envisioned speech recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Brigham Katharine, Vijaya Kumar BVK (2010) Imagined speech classification with EEG signals for silent communication: a preliminary investigation into synthetic telepathy. In: 4th international conference on bioinformatics and biomedical engineering, pp 1–4

  2. Pineda JA, Allison BZ, Vankov A (2000) The effects of self-movement, observation, and imagination on/spl mu/rhythms and readiness potentials (RP’s): toward a brain-computer interface (BCI). IEEE Trans Rehabil Eng 8 (2):219–222

    Article  Google Scholar 

  3. Jara AJ, Lopez P, Fernandez D, Castillo JF, Zamora MA, Skarmeta AF (2014) Mobile discovery: discovering and interacting with the world through the internet of things. Pers Ubiquit Comput 18(2):323–338

    Article  Google Scholar 

  4. Han K, Kim J, Shon T, Ko D (2013) A novel secure key paring protocol for rf4ce ubiquitous smart home systems. Pers Ubiquit Comput 17(5):945–949

    Article  Google Scholar 

  5. Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Pers Ubiquit Comput 18(1):19–26

    Article  Google Scholar 

  6. Pei X, Hill J, Schalk G (2012) Silent communication: toward using brain signals. IEEE Pulse 3(1):43–46

    Article  Google Scholar 

  7. Kaur B, Singh D, Roy PP A novel framework of EEG-based user identification by analyzing music-listening behavior. Multimedia Tools and Applications 1–22. https://doi.org/10.1007/s11042-016-4232-2

  8. Badcock NA, Mousikou P, Mahajan Y, de Lissa P, Thie J, McArthur G (2013) Validation of the emotiv EPOC®; EEG gaming system for measuring research quality auditory ERPs. PeerJ 1:e38

    Article  Google Scholar 

  9. Kumar P, Saini R, Roy PP, Dogra DP (2017) A bio-signal based framework to secure mobile devices. J Netw Comput Appl 89:62–71

    Article  Google Scholar 

  10. Gandhi T, Panigrahi BK, Anand S (2011) A comparative study of wavelet families for EEG signal classification. Neurocomputing 74(17):3051–3057

    Article  Google Scholar 

  11. Matsumoto M, Hori J (2014) Classification of silent speech using support vector machine and relevance vector machine. Appl Soft Comput 20:95–102

    Article  Google Scholar 

  12. Houde JF, Nagarajan SS, Sekihara K, Merzenich MM (2002) Modulation of the auditory cortex during speech: an MEG study. J Cogn Neurosci 14(8):1125–1138

    Article  Google Scholar 

  13. Price CJ (2012) A review and synthesis of the first 20years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage 62(2):816–847

    Article  Google Scholar 

  14. Kanjo E, Al-Husain L, Chamberlain A (2015) Emotions in context: examining pervasive affective sensing systems, applications, and analyses. Pers Ubiquit Comput 19(7):1197–1212

    Article  Google Scholar 

  15. Peng H, Bin H, Zheng F, Fan D, Zhao W, Chen X, Yang Y, Cai Q (2013) A method of identifying chronic stress by EEG. Pers Ubiquit Comput 17(7):1341–1347

    Article  Google Scholar 

  16. Menezes MLR, Samara A, Galway L, SantAnna A, Verikas A, Alonso-Fernandez F, Wang H, Bond R (2017) Towards emotion recognition for virtual environments: an evaluation of EEG features on benchmark dataset. Pers Ubiquit Comput 1–11. https://doi.org/10.1007/s00779-017-1072-7

  17. Costa EJX, Cabral EF (2000) EEG-based discrimination between imagination of left and right hand movements using adaptive gaussian representation. Med Eng Phys 22(5):345–348

    Article  Google Scholar 

  18. DaSalla CS, Kambara H, Sato M, Koike Y (2009) Single-trial classification of vowel speech imagery using common spatial patterns. Neural Netw 22(9):1334–1339

    Article  Google Scholar 

  19. Parra LC, Spence CD, Gerson AD, Sajda P (2003) Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring. IEEE Trans Neural Syst Rehabil Eng 11 (2):173–177

    Article  Google Scholar 

  20. D’Zmura M, Deng S, Lappas T, Thorpe S, Srinivasan R (2009) Toward EEG sensing of imagined speech. In: In International Conference on Human-Computer Interaction, pp 40–48

  21. Li W, Zhang X, Zhong X, Zhang Y (2013) Analysis and classification of speech imagery EEG for BCI. Biomed Signal Process Control 8(6):901–908

    Article  Google Scholar 

  22. Hsu Y-L, Yang Y-T, Wang J-S, Hsu C-Y (2013) Automatic sleep stage recurrent neural classifier using energy features of EEG signals. Neurocomputing 104:105–114

    Article  Google Scholar 

  23. He SL, Gao X, Yang F, Gao S (2003) Imagined hand movement identification based on spatio-temporal pattern recognition of EEG. In: 1st EMBS conference on neural engineering, pp 599– 602

  24. Deng S, Srinivasan R, Lappas T, D’Zmura M (2010) EEG classification of imagined syllable rhythm using hilbert spectrum methods. J Neural Eng 7(4). https://doi.org/10.1088/1741-2560/7/4/046006

  25. Esfahani ET, Sundararajan V (2012) Classification of primitive shapes using brain–computer interfaces. Comput Aided Des 44(10):1011–1019

    Article  Google Scholar 

  26. Torres-García AA, Reyes-García CA, Villaseñor-Pineda L, García-Aguilar G (2016) Implementing a fuzzy inference system in a multi-objective EEG channel selection model for imagined speech classification. Expert Systems with Applications 59:1–12

    Article  Google Scholar 

  27. González-Castañeda EF, Torres-García AA, Reyes-García CA, Villaseñor-Pineda L (2017) Sonification and textification: Proposing methods for classifying unspoken words from EEG signals. Biomed Signal Process Control 37:82–91

    Article  Google Scholar 

  28. Wang K, Wang X, Li G (2017) Simulation experiment of bci based on imagined speech EEG decoding. arXiv:1705.07771

  29. Nguyen CH, Karavas G, Artemiadis P (2017) Inferring imagined speech using EEG signals: a new approach using riemannian manifold features J Neural Eng https://doi.org/10.1088/1741-2552/aa8235

  30. Soleymani M, Pantic M, Pun T (2012) Multimodal emotion recognition in response to videos. IEEE Trans Affect Comput 3(2):211–223

    Article  Google Scholar 

  31. Gauba H, Kumar P, Roy PP, Singh P, Dogra DP, Raman B (2017) Prediction of advertisement preference by fusing EEG response and sentiment analysis. Neural Netw 92:77–88

    Article  Google Scholar 

  32. Donos C, Dümpelmann M, Schulze-Bonhage A (2015) Early seizure detection algorithm based on intracranial EEG and random forest classification. Int J Neural Syst 25(05):1550023

    Article  Google Scholar 

  33. Fraiwan L, Lweesy K, Khasawneh N, Wenz H, Dickhaus H (2012) Automated sleep stage identification system based on time–frequency analysis of a single EEG channel and random forest classifier. Comput Methods Prog Biomed 108(1):10–19

    Article  Google Scholar 

  34. Menze BH, Kelm BM, Masuch R, Himmelreich U, Bachert P, Petrich W, Hamprecht FA (2009) A comparison of random forest and its Gini importance with standard chemometric methods for the feature selection and classification of spectral data. BMC Bioinf 10(1):213

    Article  Google Scholar 

  35. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  36. Übeyli ED (2009) Combined neural network model employing wavelet coefficients for EEG signals classification. Digital Signal Process 19(2):297–308

    Article  Google Scholar 

  37. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

    Article  MATH  Google Scholar 

  38. Yadava M, Kumar P, Saini R, Roy PP, Dogra DP (2017) Analysis of EEG signals and its application to neuromarketing. Multimedia Tools and Applications 76(18):19087–19111

    Article  Google Scholar 

  39. Kumar P, Gauba H, Roy PP, Dogra DP (2017) A multimodal framework for sensor based sign language recognition. Neurocomputing 259:21–38

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, IIT Roorkee, Roorkee, India

    Pradeep Kumar, Rajkumar Saini, Partha Pratim Roy & Pawan Kumar Sahu

  2. School of Electrical Sciences, IIT Bhubaneswar, Bhubaneswar, India

    Debi Prosad Dogra

Authors
  1. Pradeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajkumar Saini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Partha Pratim Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pawan Kumar Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Debi Prosad Dogra
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pradeep Kumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, P., Saini, R., Roy, P.P. et al. Envisioned speech recognition using EEG sensors. Pers Ubiquit Comput 22, 185–199 (2018). https://doi.org/10.1007/s00779-017-1083-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-017-1083-4

Keywords

Search

Navigation